LTGC: Long-Tail Recognition via leveraging Generated Content

Abstract

Long-tail recognition is challenging because it requires the model to learn good representations from tail categories and address imbalances across all categories. In this paper, we propose a novel generative and fine-tuning framework, LTGC, to handle long-tail recognition via leveraging generated content.

Firstly, inspired by the rich implicit knowledge in large-scale models (e.g., large language models, LLMs), LTGC leverages the power of these models to parse and reason over the original tail data to produce diverse tail-class content.

Methodology

We propose several novel designs for LTGC to ensure the quality of the generated data and to efficiently fine-tune the model using both the generated and original data:

1. Content Generation: Leveraging large language models to generate diverse and accurate tail-class content based on parsing and reasoning over original tail data.

2. Quality Assurance: Novel mechanisms to ensure the quality and accuracy of the generated data.

3. Efficient Fine-tuning: Optimized fine-tuning strategy that effectively combines both generated and original data for improved model performance.

Experimental Results

The visualization demonstrates the effectiveness of the generation module in LTGC, which produces accurate and diverse tail data. Additionally, the experimental results demonstrate that our LTGC outperforms existing state-of-the-art methods on popular long-tailed benchmarks.

Our approach successfully addresses the fundamental challenges in long-tail recognition by combining the power of large language models with carefully designed generation and fine-tuning strategies.